Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges sampling from parameter posterior of a network via MCMC. Such culminate to lack convergence posterior. Nevertheless, this shows that nonconverged chain, generated MCMC space network, can yield marginalization valuable predictive distribution output network. Classification examples based on multilayer perceptrons showcase highly accurate distributions. The postulate limited scope for developments BNNs is partially valid; an asymptotically exact seems less plausible, yet tenable research avenue.